Applicant : Gregg D. Wilensky Attorney's Docket No.: 07844-0558US1 / P513 

Serial No. : 10/526,287 

Filed : March 1, 2005 

Page : 2 of 9 



REMARKS 

Claims 60-80 were pending as of the action mailed on July 23, 2009. Claims 60, 69, 70, 
71, and 80 are in independent form. 

Section 102 Rejections 

Claims 60-61, 67-68, 71-72, and 78-80 were rejected under 35 U.S.C. § 102(a) as 
allegedly being anticipated by Kresimir Matkovic et al., "Visual Image Query," ACM, July 1 1- 
13, 2002, pages 116-123 ("Matkovic"). 

Claims 60. 71. and 80 

The examiner rejected independent claim 60 as allegedly anticipated by Matkovic. Claim 

60 recites "calculating by a computer image parameters for each reference image wherein the 

image parameters characterize color, texture and shape features that are common to the reference 

image and at least one other reference image " and " combining by the computer the calculated 

image parameters to generate a composite reference image." The examiner asserts that Matkovic 

teaches these features. The relied-upon portion of Matkovic states that: 

The color depth of the image is reduced first. This is necessary to reduce errors 
caused by the inability to reproduce colors faithfully. After the color depth is 
reduced a limited number of rectangles of various sizes is placed in an image, the 
average color of each rectangle in CIE XYZ color space is computed, and finally 
converted to CIE LUV color coordinates . The series of LUV coordinates is part of 
the descriptor of an image, (page 1 17, 3: Main Idea, lines 5-11) (emphasis added) 

The relied-upon portion of Matkovic does not teach or suggest calculating parameters 
that characterize color, texture and shape features that are common to multiple reference images 
or generating a composite reference image from the common parameters of multiple reference 
images. Rather, the relied-upon portion of Matkovic describes processing only the color features 
of individual images to create a descriptor of each image. 

The applicant submits that claim 60 is allowable for at least this reason. Independent 
claims 71 and 80 recite similar features and are allowable for the reasons that apply to claim 60. 
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Claims 61 and 72 

The examiner rejected claim 61 as allegedly anticipated by Matkovic. Claim 61 depends 

from claim 60 and is allowable for at least the reasons set forth above with respect to claim 60. 

Claim 61 is also allowable for at least the following reason. Claim 61 recites reference images 

that are ranked and that calculating parameters for each reference image includes weighting the 

parameters based on each reference image's rank . The examiner asserts that Matkovic teaches 

this feature. The paragraph that includes the relied-upon portion of Matkovic states that: 

When the query image descriptor is compared with the target image descriptor , 
color differences are computed for each rectangle, using the CIE LUV color 
difference formula. These differences will be weighted according to the rectangle 
size using the contrast sensitivity function. In this way the differences that are 
more visible to us will be weighted stronger, and they will contribute more to the 
final distance. CIE LUV space was chosen as it is perceptually more uniform 
than CIE XYZ. For details about color spaces and conversions see[18, 9]. If 
there is some noise in the image, it will automatically be neglected by using the 
contrast sensitivity. Actually, more visible differences will contribute to the error 
more significantly, (page 117, 3: Main Idea, lines 15-25) (emphasis added) 

The relied-upon portion of Matkovic does not teach or suggest this feature. Rather, 
Matkovic teaches weighing differences in comparing a query image to a target image, with no 
mention of ranking reference images or calculating parameters based on such ranks. 

The applicant submits that claim 61 is allowable for at least these additional reasons. 
Claim 72 is allowable for the reasons that apply to claim 61. 

Claims 67. 68. 78. and 79 

Claims 67 and 68 depend from claim 60 and are allowable for at least the reasons set 
forth above with respect to claim 60. Claims 78 and 79 depend from claim 71 and are allowable 
for at least the reasons set forth above with respect to claim 71, which corresponds to claim 60. 

Section 103 Rejections 

Claims 62-64 and 73-75 were rejected under 35 U.S.C. § 103(a) as allegedly 
unpatentable over Matkovic and Essam El-Kwae et al., "A Robust Framework for Content-Based 
Retrieval by Spatial Similarity in Image Databases," ACM, 1999, pages 174-198 ("El-Kwae"). 
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Claims 65-66 and 76-77 were rejected under 35 U.S.C. § 103(a) as allegedly 
unpatentable over Matkovic, El-Kwae, and U.S. Patent No. 5,835,667 ("Wactlar"). 

Claims 69-70 were rejected under 35 U.S.C. § 103(a) as allegedly unpatentable over 
Matkovic and Wactlar. 

Claims 62-64 and 73-75 

Claims 62-64 depend from claim 60 and claims 73-75 depend from claim 71, which 
have already been addressed. As explained above, Matkovic fails to disclose calculating 
parameters that characterize color, texture and shape features that are common to multiple 
reference images, or generating a composite reference image from the common parameters of 
multiple reference images. The relied-upon portions of El-Kwae fail to remedy these 
deficiencies. Because the examiner has failed to find all the claim features in the prior art, 
claims 62-64 and 73-75 should be allowed. 

Claim 64 is also allowable for at least the following reason. The examiner rejected claim 

64 as allegedly unpatentable over Matkovic in view of El-Kwae. The examiner stated that: 

Matkovic does not explicitly teach audio objects as claimed. However, El-Kwae 
teaches the collection includes audio objects (i.e. may be implemented with 
audiovisual search engines, page 178, line 15)). 

The applicant respectfully disagrees. The sentence containing the relied-upon portion of 

El-Kwae states that: 

The standard is intended to facilitate the future development of audiovisual 
content-based search engines, (page 178, lines 14-16) 

The applicant submits that the relied-upon portion of El-Kwae does not teach or suggest 
that a collection of images includes audio objects, in which the collection is compared to a 
composite reference image generated from the common parameters of multiple reference images. 
Rather, it simply mentions the future development of "audiovisual content-based search 
engines." 

Claim 64 recites features not taught by the relied-upon portions of either Matkovic or El- 
Kwae. The applicant submits that claim 64 is allowable for at least this reason. Claim 75 is 
allowable for the reasons that apply to claim 64. 
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Claims 65-66 and 76-77 

Claims 65-66 depend from claim 60 and claims 76-77 depend from claim 71, which 
have already been addressed. As explained above, Matkovic fails to disclose calculating 
parameters that characterize color, texture and shape features that are common to multiple 
reference images, or generating a composite reference image from the common parameters of 
multiple reference images. El-Kwae and Wactlar fail to remedy these deficiencies. Because the 
examiner has failed to find all the claim features in the prior art, claims 65-66 and 76-77 should 
be allowed. 

Claim 65 is also allowable for at least the following reason. The examiner rejected claim 

65 as allegedly unpatentable over Matkovic in view of El-Kwae and Wactlar. Claim 65 recites 

" calculating sound parameters for each reference audio object wherein the sound parameters 

characterize sound features in the reference sound object that map to image parameters of at least 

one reference image ." The examiner asserts that Matkovic, El-Kwae, and Wactlar teach this 

feature. The relied-upon portion of Wactlar states that: 

The method includes the steps of transcribing audio data, marking the transcribed 
audio data with a first set of time-stamps and indexing the transcribed audio data. 
The steps of digitizing the video data and marking the digitized video data with a 
second set of time-stamps related to the first set of time-stamps are performed, 
prior to segmenting the digitized video data into paragraphs according to a set of 
rules. (Col. 4, lines 34-41) (emphasis added) 

Wactlar describes transcribing audio data into text and digitizing video data with a set of 
time-stamps. The relied-upon portion of Wactlar does not teach or suggest calculating sound 
parameters or sound parameters that map to at least one reference image . 

Claim 65 also recites " combining the sound parameters with the image parameters to 

produce composite reference information ." The examiner asserts that Matkovic, El-Kwae, and 

Wactlar teach this feature. The relied-upon portion of Wactlar states that: 

The resultant digital library 36 includes indexed, text transcripts of audio data 38, 
and segmented, compressed, audio/video data 40. The digital library may also 
include indexed text and segmented compressed video data. The digital library 36 
is the output of the offline portion 12 of the digital video library system 10. It is 
the digital library 36 which is used by the online portion 14 and which, in a 
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commercial environment, is accessed or otherwise made available to users. (Col. 
6, lines 45-54) 

The relied-upon portion of Wactlar describes an indexed "digital library" that simply 
"includes . . . audio/video data." The applicant submits that the relied-upon portion of Wactlar 
does not teach or suggest combining sound and image parameters to produce composite 
reference information. 

Claim 65 recites features not taught by Matkovic, El-Kwae, or Wactlar. The applicant 
submits that claim 65 is allowable for at least this reason. Claim 76 is allowable for the reasons 
that apply to claim 65. 

Claims 69-70 

Claim 69 

The examiner rejected independent claim 69 as allegedly unpatentable over Matkovic in 

view of Wactlar. Claim 69 recites "calculating by a computer video parameters for each 

reference video object wherein the video parameters characterize image and audio features in the 

reference video object that arc common to the reference video object and at least one other 

reference video object ." The examiner asserts that Matkovic and Wactlar teach this feature. The 

relied-upon portion of Wactlar states that: 

The method includes the steps of transcribing audio data, marking the transcribed 
audio data with a first set of time-stamps and indexing the transcribed audio data. 
The steps of digitizing the video data and marking the digitized video data with a 
second set of time-stamps related to the first set of time-stamps are performed, 
prior to segmenting the digitized video data into paragraphs according to a set of 
rules. (Col. 4, lines 34—41) 

Wactlar describes transcribing audio data into text and digitizing video data with a set of 
time-stamps. The relied-upon portion of Wactlar does not teach or suggest characterizing image 
and audio features in reference video, or video parameters that are common to multiple reference 
video objects . 

Claim 69 also recites " combining the video parameters to generate composite reference 

information ." The relied-upon portion of Wactlar states that: 

The resultant digital library 36 includes indexed, text transcripts of audio data 38, 
and segmented, compressed, audio/video data 40. The digital library may also 
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include indexed text and segmented compressed video data. The digital library 36 
is the output of the offline portion 12 of the digital video library system 10. It is 
the digital library 36 which is used by the online portion 14 and which, in a 
commercial environment, is accessed or otherwise made available to users. (Col. 
6, lines 45-54) 

The relied-upon portion of Wactlar describes an indexed "digital library" that simply 
"includes . . . audio/video data." The applicant submits that the relied-upon portion of Wactlar 
does not teach or suggest combining common video parameters from multiple reference videos 
to generate composite reference information. 

The applicant submits that claim 69 is allowable for at least these reasons. 

Claim 70 

The examiner rejected independent claim 70 as allegedly unpatentable over Matkovic in 
view of Wactlar. Claim 70 recites " calculating by a computer text parameters for each reference 
text object wherein the text parameters characterize language features in the reference text object 
that arc common to the reference text object and at least one other reference text object ." The 
examiner asserts that Matkovic and Wactlar teach this feature. The relied-upon portion of 
Wactlar states that: 

Raw material could also include pure text, audio only, or video only. 

The audio data 18 is subjected to the functions of speech and language 
interpretation 28 and speech and language indexing 30, each of which will be 
described in greater detail herein. The video data 20 is subjected to the functions 
of video segmentation 32 and video compression 34, which will also be described 
in greater detail herein. The resultant digital library 36 includes indexed, text 
transcripts of audio data 38, and segmented, compressed, audio/video data 40. 
The digital library may also include indexed text and segmented compressed 
video data. The digital library 36 is the output of the offline portion 12 of the 
digital video library system 10. It is the digital library 36 which is used by the 
online portion 14 and which, in a commercial environment, is accessed or 
otherwise made available to users. (Col. 6, lines 39-54) 

The relied-upon portion of Wactlar does not teach or suggest calculating text parameters 
that characterize language features in reference text, or text parameters that are common to 
multiple reference text objects . Rather, it simply describes that the data of its digital library may 



include pure text and indexed text. 
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Claim 70 also recites " combining by the computer the text parameters to generate 

composite reference text ." The relied-upon portion of Wactlar states that: 

Image understanding plays a critical role in organizing, searching, and reusing 
digital video. The digital video library system 10 must annotate digital video 
automatically by speech and language understanding, as well as by using other 
textual data that has been associated with the video. Spoken words or sentences 
should be attached to their associated frames. The traditional database search by 
keywords, where images are only referenced, but not directly searched for, is not 
appropriate or useful for our digital library system 10. Rather, digital video itself 
must be segmented, searched for, manipulated, and presented for similarity 
matching, parallel presentation, and context sizing while preserving image 
content. (Col. 7, lines 21-34) 

The relied-upon portion of Wactlar does not teach or suggest combining common text 
parameters from multiple reference texts to generate composite reference text. Rather, Wactlar 
describes text that annotates digital video data. 



The applicant submits that claim 70 is allowable for at least these reasons. 
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Conclusion 

For the foregoing reasons, the applicant submits that all the claims are in condition for 
allowance. 

By responding in the foregoing remarks only to particular positions taken by the 
examiner, the applicant does not acquiesce with other positions that have not been explicitly 
addressed. In addition, the applicant's selecting some particular arguments for the patentability 
of a claim should not be understood as implying that no other reasons for the patentability of that 
claim exist. Finally, the applicant's decision to amend or cancel any claim should not be 
understood as implying that the applicant agrees with any positions taken by the examiner with 
respect to that claim or other claims. 

Please apply any other charges or credits to deposit account 06-1050. 

Respectfully submitted, 



Date:October 8, 2009 /Daniel J. Burns/. 

Daniel J. Burns 
Reg. No. 50,222 
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